PLAYBACK APPARATUS AND PLAYBACK METHOD 



BACKGROUND OF THE INVENTION 
(1) Field of the Invention 
5 This invention relates to an apparatus and method 

for playing back a plurality of pieces of video 
information, more particularly, to an apparatus and method 
for receiving and playing back a plurality of pieces of 
video information delivered via a network. 
10 (2) Description of the Related Art 

In recent years, streaming delivery of content, 
such as music and animations, has been performed 
extensively via a network, such as the Internet. 

For example, to insert a commercial film into 
15 content streaming delivery of which is being made, a 
plurality of pieces of video information (video sources) 
may be continuously played back. 

Furthermore, a plurality of streaming videos have 
been continuously played back in multimedia presentation 
20 by, for example, the synchronized multimedia integration 
language (SMIL) , being a World-Wide Web Consortium (W3C) 
standard, learning systems (e-Learning) using a network, 
and the like. 

Conventional streaming video delivery in which a 
25 plurality of video sources are continuously played back 
via a network will now be described. 

Fig. 11 is a view showing the rough structure of a 
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conventional streaming video delivery system. 

This conventional streaming video delivery system 
comprises a client (playback apparatus) 500, a network 600, 
a content server 700, and a streaming server 800. 
5 The playback apparatus 500 includes a communication 

handling section 510, a scenario management section 520, a 
decoder module 530 including a streaming buffer (not 
shown) and a decoder (not shown) , and an output section 
540. 

10 In Fig. 11, the flow of data and the flow of a 

control signal are shown by solid and dotted lines 
respectively . 

A playback scenario S10 stored in the content 
server 700 is sent to the communication handling section 

15 510 in the playback apparatus 500 via the network 600 in 
accordance with a request to deliver a scenario . The order 
in which video sources V m (m=l , ... , M) are played back and 
the like are described in the playback scenario S10, which 
corresponds to, for example, an ASX file based on the 

20 Windows Media Technology developed by Microsoft 
Corporation for processing media. The scenario management 
section 520 informs the streaming server 800 of a 
streaming video delivery request in accordance with the 
order of video playback described in the playback scenario 

25 S10. After being stored temporarily in the streaming 
buffer (not shown) in the decoder module 530 used for 
performing stable video playback by alleviating the 
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instability of the network 600 as a bit stream 
corresponding to, for example, several seconds to several 
tens of seconds, the video source V m delivered from the 
streaming server 800 are processed in order of delivery by 
5 the decoder (not shown) , are output ted from the output 
section 540, and are displayed on a display device or the 
like. 

When it is time to switch the video source V m 
described in the playback scenario S10, the scenario 

10 management section 520 exerts control over the decoder 
module 530 to terminate playback. As a result, the decoder 
module 530 performs a termination process, such as erasing 
data stored in the streaming buffer (not shown) and 
stopping the decoder. After the termination process being 

15 completed, the scenario management section 520 makes a 
request to the streaming server 800 for streaming delivery 
of the next video source V m and gives the decoder module 
530 instructions to begin playback. 

From the viewpoint of a time axis , the state of 

20 this playback by the decoder module 530 is as follows. 

Fig. 12 is a view for describing the operation of a 
decoder module performed at the time of continuously 
playing back a plurality of video sources by a 
conventional method. 

25 When the video sources V x and V 2 are continuously 

played back, a termination process is performed first 
after the playback of the video source Vi being completed, 
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then an initialization process including communicating 
with the streaming server 800 and buffering a bit stream 
is performed, and then the playback of the video source V 2 
is begun. 

5 By the way, to repeatedly play back a video source 

from a specified position, an information playback 
apparatus for seamlessly displaying images without harming 
the continuity of an animation is disclosed (for example, 
see Japanese Unexamined Patent Publication No. 2001-203977, 
10 paragraph numbers [0017] - [0040] and Fig. 1). 

The information playback apparatus disclosed in 
Japanese Unexamined Patent Publication No. 2001-203977 
includes two decoders . One decoder plays back a video 
source. The other decoder plays back the video source 
15 which has repeatedly been played back, and puts it into a 
pause state. When instructions to repeatedly play back are 
given, the decoder which outputs an image will be changed. 
This enables a continuous display of a video source. 

With the conventional playback apparatus, however, 
20 time taken to perform the termination process and 
initialization process as shown in Fig. 12 corresponds to 
a non-display interval where a video image will not be 
outputted. That is to say, a video cannot be continuously 
played back. 

25 Time taken to perform buffering included in the 

initialization process has been shortened significantly by 
improving a delivery system, but it is difficult to reduce 
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other factors in the time lag. 

Moreover, with the information playback apparatus 
disclosed in Japanese Unexamined Patent Publication No. 
2001-203977, a plurality of decoders are used for 
5 continuously playing back a video source from the position 
from which repeated playback is specified. However, the 
position from which repeated playback is performed is 
determined on the basis of designation by a user. That is 
to say, this information playback apparatus cannot realize 
10 the automatic continuous playback of a plurality of video 
sources on the basis of a playback scenario inputted. 



SUMMARY OF THE INVENTION 
The present invention was made under the background 
15 circumstances as described above. An object of the present 
invention is to provide a playback apparatus which can 
continuously play back a plurality of streaming videos 
without non-display time. 

Another object of the present invention is to 
20 provide a playback method by which a plurality of 
streaming videos can be continuously played back without 
non-display time. 

In order to achieve the above first object, a 
playback apparatus for receiving and playing back a 
25 plurality of pieces of video information delivered via a 
network is provided. This playback apparatus comprises a 
plurality of decoder modules for decoding the plurality of 
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pieces of video information, a scenario management section 
for reading a playback scenario where playback information 
regarding the plurality of pieces of video information is 
described, for determining a playback schedule according 
5 to the playback information, and for exercising 
distribution of the plurality of pieces of video 
information among the plurality of decoder modules and 
switching control over output from the plurality of 
decoder modules, an output switching section for switching 

10 output from the plurality of decoder modules under the 
switching control, a communication handling section for 
making a request via the network to deliver the playback 
scenario, receiving the playback scenario via the network, 
making a request via the network to deliver the plurality 

15 of pieces of video information, and receiving the 
plurality of pieces of video information via the network, 
and an output section for outputting the plurality of 
pieces of video information. 

In order to achieve the above second object, a 

20 playback method for receiving and playing back a plurality 
of pieces of video information delivered via a network is 
provided. This playback method comprises the steps of 
reading a playback scenario where playback information 
regarding the plurality of pieces of video information is 

25 described, determining a playback schedule according to 
the playback information, and exercising, in accordance 
with the playback schedule, distribution of the plurality 
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of pieces of video information among a plurality of 
decoder modules and switching control over output from the 
plurality of decoder modules . 

The above and other objects, features and 
advantages of the present invention will become apparent 
from the following description when taken in conjunction 
with the accompanying drawings which illustrate preferred 
embodiments of the present invention by way of example . 

BRIEF DESCRIPTION OF THE DRAWINGS 

Fig. 1 is a view for describing the principles 
underlying a playback apparatus according to the present 
invention and the structure thereof. 

Fig. 2 is a view for describing data structure 
indicative of a video source . 

Fig. 3 is a flow chart showing a method for 
distributing M video sources among N decoder modules . 

Fig. 4 is a view showing an example in which the 
distribution method shown in Fig. 3 is applied. 

Fig. 5 is a view for describing the timing with 
which a decoder module is controlled. 

Fig . 6 is a view for describing the timing with 
which a decoder module is controlled in the case of a 
delay in initialization exceeding a margin. 

Fig. 7 is a view showing the structure of a 
playback apparatus according to an embodiment of the 
present invention . 



Fig. 8 is the state transition diagram of a 
synchronization controller. 

Fig. 9 is the state transition diagram of a video 

player . 

Fig. 10 is a flow chart showing a method for 
distributing M video sources among N decoder modules in 
dynamic scheduling. 

Fig. 11 is a view showing the rough structure of a 
conventional streaming video delivery system. 

Fig. 12 is a view for describing the operation of a 
decoder module performed at the time of continuously 
playing back a plurality of video sources by a 
conventional method. 

DESCRIPTION OF THE PREFERRED EMBODIMENT 

An embodiment of the present invention will now be 
described with reference to the drawings . 

Fig. 1 is a view for describing the principles 
underlying a playback apparatus according to the present 
invention and the structure thereof. 

A playback apparatus 10 according to the present 
invention is connected via a network 20, such as the 
Internet, to a content server 30 which manages a playback 
scenario SI and a streaming server 40 which manages a 
plurality of video sources V m (m=l , ... , M) . 

In Fig. 1, the flow of data and the flow of a 
control signal are shown by solid and dotted lines 
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respectively . 

The playback apparatus 10 comprises decoder modules 
D n (n=l, ... , N) , a scenario management section 11, a 
changeover switch 12 , a communication handling section 13, 
5 and an output section 14. 

Each of the decoder modules D n includes a streaming 
buffer (not shown) and a decoder (not shown) and decodes 
an encoded streaming video. 

The scenario management section 11 reads the 
10 playback scenario SI where playback information regarding 
the plurality of video sources V m is described , determines 
a playback schedule according to the playback information, 
and exercises distribution of the video sources V m among 
the decoder modules D n and switching control over output 
15 from the decoder modules D n . 

The changeover switch 12 is used for switching 
output from the decoder modules D n under switching control 
exercised by the scenario management section 11 . 

The communication handling section 13 makes a 
20 request via the network 20 to deliver the playback 
scenario SI, receives the playback scenario SI via the 
network 20, makes a request via the network 20 to deliver 
the video sources V m , and receives the video sources V m via 
the network 20 . 

25 The output section 14 outputs the video sources V m 

decoded by the decoder module D n selected by the changeover 
switch 12 to a display (not shown) or the like. 
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The operation of the playback apparatus 10 will now 
be described. 

When the playback apparatus 10 makes a request to 
deliver the playback scenario SI, the playback scenario SI 
5 stored in the content server 30 is inputted to the 
communication handling section 13 via the network 20 and 
is downloaded into the scenario management section 11 . The 
scenario management section 11 reads and analyzes the 
playback scenario SI, extracts the video sources V m to be 

10 displayed in succession in a content and the time when the 
video sources V m are to be displayed from playback 
information described in the playback scenario SI, and 
rearranges them in order of display time. Then the 
scenario management section 11 determines the distribution 

15 of the video sources V m among the plurality of decoder 
modules D n , a playback schedule, and a schedule of 
switching by the changeover switch 12 on the basis of a 
time lag (AS) in displaying the video sources V m managed by 
the streaming server 40 and time (AE) taken to perform a 

20 termination process . 

Table 1 shows data structure indicative of a video 

source . 

25 
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URL 


place where video 
source itself is stored 1 


start 


playback s tar ting time 
indicated by general time 


local start 


playback starting time 
indicated by video time 


duration 


video playback time 


decoder 


ID of decoder to which 
video source is distributed 



Table 1 



As shown in Table 1 , data structure indicative of 
the video source V m includes "URL (uniform resource 
5 locator) " which indicates a place where the video source V m 
itself is stored, "start" which indicates playback 
starting time by general time (scenario time) , being the 
time when a user is watching or listening, "locales tart" 
which indicates playback starting time by video time, 
10 being time in the video source V m , "duration" which 
indicates video playback time, and "decoder" which 
indicates the ID of a decoder to which the video source V m 
is distributed. "URL," "start," "local_start , " and 

"duration" are described in a playback scenario and 
15 "decoder" is added to the data structure by the scenario 
management section 11 by the method described later. 

Fig. 2 is a view for describing data structure 
indicative of a video source. 

As shown in Fig. 2, when a user watches, a playback 
20 interval in the video source V m specified by 
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"V m . local_start" and "V m . duration" by the video time will 
be incorporated into time specified by "V m . start" by the 
scenario time. It is assumed that the M video sources V m 
have been sorted in ascending order of "start". 
5 At this time the plurality of video sources V m will 

be distributed among the decoder modules D n in the 
following way. 

Fig. 3 is a flow chart showing a method for 
distributing M video sources among N decoder modules . 

10 Descriptions will now be given with video source 

IDs as m (m=l,... , M) and with decoder IDs as n (n=l,... , N) . 

When the scenario management section 11 reads the 
playback scenario SI, the scenario management section 11 
sets the initial value of m, being the ID of distributed 

15 video sources V m . With static distribution in which all of 
the video sources V m are distributed before a content being 
played back, the initial value of m is set to 1. With 
dynamic distribution in which a playback scenario is added 
with the playback of a content or according to 

20 circumstances, the initial value of m is set to the 
smallest one of the IDs of video sources which are not yet 
distributed (step SI) . 

Next, the video source V m . which has already been 
distributed and which meets the condition (condition 1) of 

25 "V m . .URL=V m .URL" and "V m . . start+V m . . dura tion=V m . start" is 
searched for. That is to say, the video source V m . which 
is stored in the same place as the video source V m and the 
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ending -time of which matches the playback starting time of 
the video source V m is searched for. If there is video 
source V m . which meets the above condition, then step S3 
will be performed. If there is no video source V m » which 
5 meets the above condition, then step S4 will be performed 
(step S2) . 

If the condition shown in step S2 is met, then the 
same decoder module that was used to play back the video 
source V m . should be used to decode the video source V m . 
10 Therefore, !l V m . decoder— V m » . decoder n is given and the ID of 
a decoder used to decode the video source V m is the same as 
the ID of the decoder used to decode the video source V m . 
(step S3) . 

If there is no video source V m . which meets the 
15 condition shown in step S2 , then decoder module D n which is 
not being used at time "V m . start" is extracted. It is 
assumed that the last ending time of the decoder module D n 
is T. Decoder module D n which meets the condition of 
11 V m . start- AS>T+AE" is searched for. That is to say, 
20 decoder module D n which meets the condition that the 
playback starting time 11 V m . start" of the video source V m 
minus time "AS" taken to perform an initialization process 
is greater than its last ending time T plus time AE taken 
to perform a termination process (the margin described 
25 later is taken into consideration) is searched for. If 
there is one decoder module D n which meets the above 
condition, then step S5 will be performed. If there are a 
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plurality of decoder modules D n which meet the above 
condition, then step S6 will be performed. If there is no 
decoder module D n which meets the above condition, then 
step S7 will be performed (step S4) . If there is only one 
5 decoder module D n which meets the above condition, it is 
used for playback. That is to say, "V m . decoder^n" is given 
(step S5) . If there are a plurality of decoder modules D n 
which meet the above condition, n V m . decoder" is set to the 
decoder ID of one of them that terminates playback first 
10 (step S6) . If there is no decoder module D n which meets 
the above condition, "V m . decoder" is set to the decoder ID 
of one of all decoder modules D n that terminates playback 
first (step S7) . When the setting of "V m . decoder" in any 
of steps S5 through S7 is completed, m, being a video 
15 source ID, is incremented by 1 (step S8) . If m>M, that is 
to say, if the distributing of all video sources V m is 
completed, then the process is terminated. If m<M, then 
the process will be repeated from step S2 (step S9) . 

By doing so, a plurality of video sources V m can be 
20 distributed among a plurality of decoder modules D n . 

Fig. 4 is a view showing an example in which the 
distribution method shown in Fig. 3 is applied. 

In this example, video sources V 8 through V i2 are 
distributed among a plurality of decoder modules D x through 
25 D 3 . 

It is assumed that video sources V m have been 
extracted and have been rearranged in order of display 



time in accordance with the playback scenario SI 
downloaded. 

It is assumed that the video source Vu having the 
video source ID of 11 is distributed. If there is no video 
5 source V m . which meets the condition of "V m - .URL=V n .URL" 
and 11 V m . . start+V m . .dura tion=Vn . start" in step S2 shown in 
Fig. 3, then step S4 will be performed. In this case, of 
decoder modules D n which are not being used, only the 
decoder module Di having the decoder ID of 1 meets the 

10 condition of "Vu . start-AS>T+AE" . Therefore, step S5 will 
be performed and the video source Vu is distributed to the 
decoder module Di to decode. 

The scenario management section 11 controls the 
changeover switch 12 in accordance with a switching 

15 schedule shown at the bottom of Fig. 4 to switch output 
from the decoder modules Di through D 3 which decode the 
video sources V 8 through V i2 distributed in this way. This 
enables a seamless video display. 

Fig. 5 is a view for describing the timing with 

20 which a decoder module is controlled. 

For example, with the decoder module Di shown in 
Fig. 4, an initialization process for the next video 
source Vu is begun some time after a termination process 
for the video source V 8 is completed. As shown in Fig. 5, 

25 however, when a termination process for the video source V 8 
is completed and the decoder module Di becomes reusable, an 
initialization process for the next video source Vu will 
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actually be begun. 

The other decoder modules D n operate in the same 

way . 

If the rules for distributing the video sources V m 
5 are followed, they should go into a state in which they 
can be played back by the time when playback is begun. 
However, initialization time includes indefinite factors, 
such as the occurrence of a network delay, so there is no 
guarantee that the video sources V m will go into a state in 
10 which they can be played back by the time when playback is 
begun. Accordingly, when a plurality of decoder modules D n 
can be used, the video sources V m are distributed to 
decoder modules D n which have not been used for a longer 
time to allow for a margin against a delay in 
15 initialization . 

Fig. 6 is a view for describing the timing with 
which a decoder module is controlled in the case of a 
delay in initialization exceeding a margin . 

If a delay in initialization exceeds a margin 
20 allowed for in advance, that is to say, if initialization 
is not completed by the time when playback is begun, the 
subsequent processes will be performed by delaying the 
entire schedule time for the time corresponding to a delay 
in beginning playback. If such a delay occurs, the 
25 subsequent video sources may be redistributed. 

A concrete embodiment of the present invention will 
now be described. 
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Fig. 7 is a view showing the structure of a 
playback apparatus according to an embodiment of the 
present invention . 

A playback apparatus 100 according to an embodiment 
of the present invention is connected to a content server 
200 including a scenario server 210 and a streaming server 
220. 

The scenario server 210 stores a playback scenario 
in which information included in the data structure 
indicative of the video sources V m shown in Table 1, 
excluding "decoder," has been described in the CSV format, 
the XML format , or the like . 

Table 2 shows an example of a playback scenario 
described in the CSV format. 

start [s] local_start[s] duration [s] URL 

0.0, 0.0, 180.0, mms://f oo01.com/bar001.wmv 

180.0, 0.0, 400.0, nuns : //f oo02 . com/bar002 . wmv 

580.0, 120.0, 600.0, mms : //f oo03 . com/bar003 . wmv 



Table 2 

One line indicates one video source V m . Pieces of 
information indicative of "start," "locales tart , " 
"duration," and "URL" are described in that order in 
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seconds and are separated by commas . 

The streaming server 220 stores various video 
contents. An existing product, such as RealServer 

(trademark of the RealNetworks Inc.) or Windows Media 
Server (trademark of the Microsoft Corporation) , will be 
used for performing streaming delivery at need. A 
plurality of servers will be used depending on a playback 
scenario . 

The playback apparatus 100 comprises a user 
interface 110, a whole control section 120, a scenario 
parser 130, a synchronization controller 140, a scheduler 
150, a video player 160, and a display section 170. 

The playback apparatus 100 may be a personal 
computer or dedicated hardware. If the playback apparatus 
100 is a personal computer, then the whole or part of the 
user interface 110, the whole control section 120, the 
scenario parser 130, the synchronization controller 140, 
the scheduler 150, and the video player 160 in the 
playback apparatus 100 may be described in a downloadable, 
executable language, such as Java or JavaScript, to 
download and execute them as each module before 
downloading a scenario. If the playback apparatus 100 is 
dedicated hardware, then they will be realized as hardware 
modules . 

The scenario server 210 and streaming server 220 
shown in Fig. 7 correspond to the content server 30 and 
streaming server 40, respectively, shown in Fig. 1. In Fig. 
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7, the scenario server 210 and streaming server 220 are 
united into the content server 200. The network 20 shown 
in Fig. 1 is omitted in Fig. 7. 

In the playback apparatus 100 shown in Fig. 7, the 
5 user interface 110 , the whole control section 120 , part of 
the function of the scenario parser 130, the 
synchronization controller 140 , and the scheduler 150 
correspond to the scenario management section 11 shown in 
Fig. 1. Part of the function of the scenario parser 130 

10 corresponds to the communication handling section 13 shown 
in Fig. 1. This communication handling section 13 is 
omitted in Fig. 7. In addition, the video player 160 and 
the display section 170 correspond to the decoder module D n 
and the output section 14, respectively. 

15 The user interface 110 enables a user to give 

instructions to specify a playback scenario to be played 
back (input URL) , instructions to play back a playback 
scenario, instructions to stop or pause the playback of a 
playback scenario, or the like. For example, a keyboard, a 

20 switch, or a mouse (not shown) will be connected to the 
user interface 110. 

The whole control section 120 is a module for 
converting instructions from a user inputted by the user 
interface 110 to operation instructions to each module. 

25 When instructions to read a playback scenario are 

given, the scenario parser 130 communicates with the 
scenario server 210, downloads the playback scenario, and 
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converts it to an internal format. For example, an array 
having the data structure indicative of the video sources 
V m (m=l , ... , M) shown in the above Table 1 can be used as an 
internal format for a playback scenario. When the scenario 
5 parser 130 converts a playback scenario to an internal 
format, the scenario parser 130 also sorts data in 
ascending order of "start, " being playback starting time 
by the general time. 

The synchronization controller 140 sets data 

10 regarding a plurality of video sources V m shown in, for 
example, Table 1 (excluding "decoder") which is inputted 
via the whole control section 120 and which has been 
sorted in ascending order, and controls the operation and 
output of the video player 160 in accordance with a 

15 playback schedule determined by the scheduler 150. 
Furthermore, the synchronization controller 140 controls 
the operation of the video player 160 in accordance with 
instructions from the whole control section 120 to play 
back a playback scenario or to stop or pause the playback 

20 of a playback scenario. 

On the basis of data regarding a plurality of video 
sources V m set in the synchronization controller 140, the 
scheduler 150 determines by the distribution method 
described in Fig. 3 which video player 160 plays back each 

25 of the plurality of video sources V m . 

There are a plurality of video players 160, which 
corresponds to the fact that a plurality of streaming 
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servers 220 may be used. Under the control of the 
synchronization controller 140, each video player 160 
temporarily stores video sources V m inputted via the 
network (not shown) in a streaming buffer (not shown) as 
5 bit streams corresponding to several seconds to several 
tens of seconds, decodes them in order of store, and 
outputs them to the display section 170. By doing so, the 
instability of the network (not shown) will be alleviated 
and stable video playback will be performed. 
10 The display section 170 displays video sources V m 

decoded by the video players 160 on a display (not shown) 
or the like . 

In this embodiment, the same effect that can be 
obtained by the use of the changeover switch 12 explicitly 

15 shown in Fig. 1 is realized by controlling the 
visibility /invisibility of all the video players 160 . 
Output images are displayed one over another by the 
display section 170. However, a playback scenario has been 
set so that one video player 160 will be visible. 

20 Therefore, only one video will always be displayed. 

The operation of the playback apparatus 100 will 
now be described. 

When the URL of a playback scenario is inputted 
from the user interface 110, the whole control section 120 

25 gives the scenario parser 130 instructions to read the 
playback scenario. The scenario parser 130 downloads the 
playback scenario from the scenario server 210, converts 
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it to an internal format, and sorts a plurality of video 
sources V m in ascending order of playback starting time by 
the general time. Data indicative of the plurality of 
video sources V m read and sorted is set as an object of 
playback by the synchronization controller 140 via the 
whole control section 120. 

The operation of the synchronization controller 140 
will now be described. 

Fig. 8 is the state transition diagram of the 
synchronization controller . 

Each of symbols Tl through T4 indicates a state. 

When the synchronization controller 140 is in an 
initial state (state Tl) and a playback scenario is read, 
data indicative of video sources V m is set as an object of 
playback and the synchronization controller 140 makes the 
transition to a state T2 . When the synchronization 
controller 140 is in the state T2 , the playback of the 
playback scenario is in a stopped state. At this time a 
periodic start turns off and the scenario time goes into 
an initial state (Ts=0.0 sec) . When the synchronization 
controller 140 is in the state T2 and the whole control 
section 120 gives instructions to play back the playback 
scenario, the synchronization controller 140 is started 
periodically every minute period At (several milliseconds, 
for example) and the playback scenario goes into a played 
back state (state T3) . Therefore, the video players 160 
begin to play back and output in accordance with a 
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schedule determined by the scheduler 150. When the whole 
control section 120 gives instructions to pause the 
playback of the playback scenario, the periodic start is 
stopped and the advance of the scenario time stops (state 
T4) . When the whole control section 120 gives instructions 
to stop the playback of the playback scenario, the 
synchronization controller 140 makes the transition to the 
state T2. Accordingly, the periodic start is stopped and 
the scenario time is returned to the initial state (Ts=0.0 
sec) . When the playback of the playback scenario is in a 
paused state (state T4) and the whole control section 120 
gives instructions to play back the playback scenario, the 
synchronization controller 140 makes the transition to the 
state T3. When the playback of the playback scenario is in 
a paused state (state T4) and the whole control section 
120 gives instructions to stop the playback of the 
playback scenario, the synchronization controller 140 
makes the transition to the state T2 . 

The playback state (state T3) will now be described 
in detail. 

As stated above, the synchronization controller 140 
is started periodically when a playback scenario is in a 
playback state. When the synchronization controller 140 is 
started periodically, time which elapsed after the 
beginning of the playback of the playback scenario is 
reflected in scenario time Ts . By comparing the scenario 
time Ts and an array of video sources V m which make up the 
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playback scenario, the operation of the video player 160 
indicated by "V m . decoder" is determined on the basis of the 
following conditions . 

If "V m _i . start+V m _i . dura tion<T s <V m . start" (condition 
5 2) , that is to say, if the playback of the video source V m _ 
i has been completed, the playback of the next video source 
V m is not yet performed, and the video player is in a 
stopped state, then "V m .URL" will be set to perform 
prefetch . 

10 If "V m . start<T s <V m . start+V n .duration M (condition 3), 

that is to say, if the video source V m _i is being played 
back, then playback will be performed for 
"V m .local_start+V m . start-T g " seconds by video local time. 

If "V m . star t+V m . dura tion<T s " (condition 4), that is 

15 to say, if the video source V m is played back, then the 
synchronization controller 140 will make the transition to 
the state T2 (the playback of the playback scenario is in 
a stopped state) . 

If m=l, then condition 2 becomes "T g <V m .start n . 

20 It is assumed that the video source V m meets 

condition 4 . If there is video source V m+ i which meets 
condition 1 described in step S2 shown in Fig. 3, then the 
playback of the next V m+ i under condition 3 will precede. 
That is to say, video sources V m having the same URL in 

25 " V m . decoder (=V m+ i. decoder ) " will be continuously played back. 

Though the transition to a prefetch state is made 
under condition 2 , condition 3 may be met before the 
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transition to a prefetch state being completed. In this 
case, playback cannot be begun. Therefore, until the 
transition to a prefetch state is completed, T s will not be 
increased from "T s =V m . start" . As soon as the transition to 
5 a prefetch state is completed, the transition to a 
playback state will be made and an increase of T s will be 
resumed. 

The operation of the video players 160 will now be 
described. 

10 Fig. 9 is the state transition diagram of the video 

players . 

By exercising control, such as playback, pausing 
playback, stopping playback, or setting a URL, over the 
video player 160 under the above condition 2, 3, or 4 , it 

15 makes state transition. 

When the video player 160 is in a stopped state 
(state T10) and fl V m .URIi" is set, the video player 160 makes 
the transition to a prefetch state (state Til) . In this 
case, the video player 160 will seek at "V m . local_start , 11 

20 being playback starting time by the video time. When 
prefetch is completed, the video player 160 makes the 
transition to a video playback enable state (state T12) . 
At this time the video player 160 is in a paused state and 
does not provide output to the display section 170. When 

25 playback control is exercised over the video player 160 in 
a video playback enable state, the video player 160 makes 
the transition to a video playback state (state T13) . When 
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stop control is exercised over the video player 160 in a 
video playback enable state, the video player 160 makes 
the transition to a stopped state. The video player 160 
outputs a video it decoded to the display section 170. 
When stop control is exercised over the video player 160 
in a video playback state, the video player 160 makes the 
transition to a stopped state. When pause control is 
exercised over the video player 160 in a video playback 
state, the video player 160 makes the transition to a 
video playback enable state. 

Moreover, the visibility/invisibility (or 

output/non-output) of a video played back by the video 
player 160 can be switched. When the video player 160 is 
in a video playback state (state T13) , a video played back 
by the video player 160 is visible. When the video player 
160 is in another state, a video played back by the video 
player 160 is invisible. 

Dynamic scheduling will now be described. 
A case where static scheduling is performed at the 
time of a playback scenario being set in the 
synchronization controller 140 has been described. With 
static scheduling, fixed values are used as the necessary 
coefficients AS and AE and individual video sources V m are 
distributed in advance among the video players 160. 
However, these values will vary according to network 
conditions, loads on the streaming servers 220 to which 
the video players 160 are connected, and the like. 
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With dynamic scheduling, the video players 160 feed 
back values actually determined in processes performed 
theretofore by them to the synchronization controller 140 
and these values are reflected in the subsequent 
5 scheduling performed by the scheduler 150. As a result, 
continuous video playback can be performed more flexibly. 

The operation of the playback apparatus 100 at 
dynamic scheduling time will now be described. 

When a playback scenario is set, the 
10 synchronization controller 140 initially operates the same, 
whether at static scheduling time or at dynamic scheduling 
time . 

The synchronization controller 140 has a table 
which stores AS and AE for each of the streaming servers 
15 220 included in URLs set in the video players 160. 

Table 3 is an example of a coefficient store table. 



server name 


AS 


AE 


f ooOl . com 


15 . Osec 


1 . Osec 


f oo02 . com 


10 . Osec 


1 . 5sec 









Table 3 



20 The fixed values used in the static scheduling will 

be used as the initial values of AS and AE in this table. 
For example, each time the synchronization controller 140 
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gives the video player 160 instructions to play back, the 
synchronization controller 140 inquires the measured value 
of time from a URL being set to the video player 160 
making the transition to a video playback enable state and 
5 the measured value of time from instructions to stop being 
given to the video player 160 making the transition to a 
stopped state and updates the values of AS and AE 
corresponding to the appropriate server name by the use of 
the newest measured values. To access each coefficient in 

10 the coefficient store table, the forms of, for example, 
AS [server name] and AE [server name] will be used. 

When the values of AS and AE are updated and values 
in the coefficient store table change, rescheduling will 
be performed. Video sources "V m *.URL" which are not yet 

15 set in the video players 160 are to be rescheduled. That 
is to say, the video sources V m *, V m *+i,..., V M will be 
rescheduled and scheduling will be performed with m* as 
the initial value of m. 

Fig. 10 is a flow chart showing a method for 

20 distributing M video sources among N decoder modules in 
dynamic scheduling. 

The flow chart shown in Fig. 10 differs from that 
shown in Fig. 3 only in steps S10 and S13 . 

As stated above, video sources "V m *.URI," which are 

25 not yet set in the video players 160 are to be rescheduled 
in step S10. That is to say, the video sources V m *, V m * +1 ,..., 
V M will be rescheduled, so the initial value of m will be 
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set to m* . 

In step S13, decoder module D n which is not being 
used is searched for. This is the same with step S4 in Fig. 
3. In this case, however, AS and AE are set according to 
5 the state of the streaming servers 220 at access time by 
the use of the condition of "V m . star t-AS [server name 
included in V m . URL] >T+AE [preceding server name]". 

The other processes are the same as those in the 
distribution method shown in Fig. 3, so descriptions of 
10 them will be omitted. 

As described above, by determining AS and AE on the 
basis of measured values obtained in processes performed 
before communication with the streaming servers 220 and by 
feeding back them to the synchronization controller 140, 
15 continuous playback in which the newest streaming state of 
the appropriate server is reflected can be realized. 

Static scheduling may be performed by the use of a 
coefficient store table in which the values of the 
coefficients for each server are fixed. In this case, 
20 scheduling is realized by using the flow chart in Fig. 10 
on which m* is set to 1 . Values measured for each 
streaming server 220 should be used as the fixed 
coefficients in the coefficient store table. In addition, 
values in this fixed coefficient store table may be used 
25 as initial values in a coefficient store table at dynamic 
scheduling time . 

In the above descriptions it has been assumed that 
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delivery of the entire playback scenario was completed 
before the playback of the playback scenario. However, the 
playback scenario can be considered as a set of a 
plurality of scenario Sp (p=l, 2,...). In this case, even 
before the entire playback scenario is delivered, partial 
scenarios Sp delivered in order can be executed. 

The scenario server 210 manages the playback 
scenario as a set of partial scenarios Sp (p=l, 2,...). 
Partial scenarios Sp may be stored in advance or may be 
generated in order at calling time. 

When the whole control section 120 gives the 
scenario parser 130 instructions to read the playback 
scenario, the scenario parser 130 begins to read the 
playback scenario and obtains a partial scenario Sp. The 
partial scenario Sp is sent to the synchronization 
controller 140, is distributed to a video decoder by 
static or dynamic scheduling, and is played back. 

The whole control section 120 gives the scenario 
parser 130 instructions every certain period of time to 
read the partial scenario Sp subsequent to the one which 
the scenario parser 130 has already received. If there is 
no partial scenario Sp read, then there will be nothing to 
be changed. If there is a partial scenario Sp read, then 
the whole control section 120 sets the partial scenario Sp 
in the synchronization controller 140. The synchronization 
controller 140 combines partial scenarios Sp which have 
already been set therein and the partial scenario Sp newly 
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set, performs dynamic scheduling on the combined partial 
scenarios Sp, and continues its playback. A scheduling 
method used in this case is the same as that shown in Fig. 
10. 

5 Partial scenarios Sp may be delivered from the 

scenario server 210 in push mode. In this case, partial 
scenarios Sp delivered in push mode are parsed by the 
scenario parser 130 and are processed in the same way that 
was described above . 

10 The above functions can be realized with a computer. 

In this case, a program in which the contents of the 
functions the playback apparatus 100 should have are 
described is provided. By executing this program on a 
computer, the above functions are realized on the computer. 

15 This program can be recorded on a computer readable record 
medium. A computer readable record medium can be a 
magnetic recording device, an optical disk, a magneto- 
optical recording medium, a semiconductor memory, or the 
like. A magnetic recording device can be a hard disk drive 

20 (HDD), a flexible disk (FD) , a magnetic tape, or the like. 
An optical disk can be a digital versatile disc (DVD) , a 
digital versatile disc random access memory (DVD-RAM) , a 
compact disc read only memory (CD-ROM) , a compact disc 
recordable (CD-R) /rewritable (CD-RW), or the like. A 

25 magneto-optical recording medium can be a magneto-optical 
disc (MO) or the like . 

To place the program on the market, portable record 
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media, such as DVDs or CD-ROMs, on which it is recorded 
are sold. Alternatively, the program is stored in advance 
on a hard disk in a server computer and is transferred to 
another computer via a network . 
5 When a computer executes this program, it will 

store the program, which is recorded on a portable record 
medium or which is transferred from a server computer, on, 
for example, its hard disk. Then it reads the program from 
its hard disk and performs processes in compliance with 
10 the program. A computer can also read the program directly 
from a portable record medium and perform processes in 
compliance with the program. Furthermore, each time the 
program is transferred from a server computer, a computer 
can perform processes in turn in compliance with the 
15 program it received. 

As has been described in the foregoing, in the 
present invention, a plurality of video sources delivered 
via a network can continuously be played back in the order 
which is described in a playback scenario without non- 
20 display time at each video joint. 

Furthermore, scheduling for distributing a 
plurality of video sources among a plurality of decoder 
modules is performed, so flexible continuous playback can 
be realized without describing a playback schedule in 
25 advance in a playback scenario. 

The foregoing is considered as illustrative only of 
the principles of the present invention. Further, since 
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numerous modifications and changes will readily occur to 
those skilled in the art, it is not desired to limit the 
invention to the exact construction and applications shown 
and described, and accordingly, all suitable modifications 
5 and equivalents may be regarded as falling within the 
scope of the invention in the appended claims and their 
equivalents . 
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